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Abstract. We study observation-based strategies for two-player turn-based games on 
graphs with omega-regular objectives. An observation-based strategy relies on imperfect 
information about the history of a play, namely, on the past sequence of observations. Such 
games occur in the synthesis of a controller that does not see the private state of the plant. 
Our main results are twofold. First, we give a fixed-point algorithm for computing the set 
of states from which a player can win with a deterministic observation-based strategy for 
any omega-regular objective. The fixed point is computed in the lattice of antichains of 
state sets. This algorithm has the advantages of being directed by the objective and of 
avoiding an explicit subset construction on the game graph. Second, we give an algorithm 
for computing the set of states from which a player can win with probability 1 with 
a randomized observation-based strategy for a Biichi objective. This set is of interest 
because in the absence of perfect information, randomized strategies are more powerful 
than deterministic ones. We show that our algorithms are optimal by proving matching 
lower bounds. 
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1. Introduction 

Two-player games on graphs play an important role in computer science. In particular, 
the controller synthesis problem asks, given a model for a plant, to construct a model for a 
controller such that the behaviors resulting from the parallel composition of the two models 
respects a given specification (e.g., are included in an w-regular set). Controllers can be 
synthesized as winning strategies in a game graph whose vertices represent the plant states, 
and whose players represent the plant and the controller |18| I17j . Other applications of 
game graphs include realizability and compatibility checking, where the players represent 
parallel processes of a system, or its environment [lj lll| IS]. 

Most results about two-player games played on graphs make the hypothesis of perfect 
information. In this setting, the controller knows, during its interaction with the plant, the 
exact state of the plant. In practice, this hypothesis is often not reasonable. For example, 
in the context of hybrid systems, the controller acquires information about the state of 
the plant using sensors with finite precision, which return imperfect information about the 
state. Similarly, if the players represent individual processes, then a process has only access 
to the public variables of the other processes, not to their private variables |19| 12]. 

Two-player games of imperfect information are considerably more complicated than 
games of perfect information. First, decision problems for imperfect-information games 
usually lie in higher complexity classes than their perfect-information counter-parts |19| 
[T4"t 12]. The algorithmic difference is often exponential, due to a subset construction that, 
similar to the determinization of finite automata, turns an imperfect-information game 
into an equivalent perfect-information game. Second, because of the determinization, no 
symbolic algorithms are known to solve imperfect-information games. This is in contrast 
to the perfect-information case, where (often) simple and elegant fixed-point algorithms 
exist \12\ Third, in the context of imperfect information, deterministic strategies are 
sometimes insufficient. A game is turn-based if in every state one of the players chooses 
a successor state. In turn-based games of perfect information the set of winning states 
coincides with the set of states where the probability of winning is 1, and so deterministic 
strategies suffice to win (and thus also to win with probability 1). In contrast, in turn- 
based games of imperfect information the set of winning states is in general a strict subset 
of the set of states where the probability of winning is 1, and so randomized strategies 
are required to win with probability 1 (see Example I2.3() . Fourth, winning strategies for 
imperfect-information games need memory even for simple objectives such as safety and 
reachability (see Example 14. 4p . This is again in contrast to the perfect-information case, 
where turn-based safety and reachability games can be won with memory less strategies. 

The contributions of this paper are twofold. First, we provide a symbolic fixed-point 
algorithm to compute winning states in games of imperfect information for arbitrary ir- 
regular objectives. The novelty is that our algorithm is symbolic; it does not carry out an 
explicit subset construction. Instead, we compute fixed points on the lattice of antichains of 
state sets. Antichains of state sets can be seen as a symbolic and compact representation for 
C-downward-closed sets of sets of statesQ This solution extends our recent result [10] from 
safety objectives to all w-regular objectives. To justify the correctness of the algorithm, we 
transform games of imperfect information into games of perfect information while preserving 

^We recently used this symbolic representation of C-downward-closed sets of state sets to propose a new 
algorithm for solving the universality problem of nondeterministic finite automata. First experiments show 
a very promising performance; see [5] for details. 
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the existence of winning strategies for every objective. The reduction is only part of the 
proof, not part of the algorithm. For the special case of parity objectives, we obtain a 
symbolic Exptime algorithm for solving parity games of imperfect information. This is 
optimal, as the reachability problem for games of imperfect information is known to be 
ExPTlME-hard [19]. 

Second, we study randomized strategies and winning with probability 1 for imperfect- 
information games. To our knowledge, for these games no algorithms (symbolic or not) are 
present in the literature. Following [7J, we refer to winning with probability 1 as almost-sure 
winning (almost winning, for short), in contrast to sure winning with deterministic strate- 
gies. We provide a symbolic Exptime algorithm to compute the set of almost-winning 
states for games of imperfect information with Biichi objectives (reachability objectives can 
be obtained as a special case, and for safety objectives almost winning and sure winning 
coincide). Our solution is again justified by a reduction to games of perfect information. 
However, for randomized strategies the reduction is different, and considerably more compli- 
cated. We prove our algorithm to be optimal, showing that computing the almost-winning 
states for reachability games of imperfect information is ExPTlME-hard. The problem of 
computing the almost- winning states for coBiichi objectives under imperfect information in 
Exptime remains an open problem. 

The paper is organized as follows. Section 2 presents the definitions; Section 3 gives the 
algorithm for the case of sure winning with deterministic strategies; Section 4, for the case 
of almost winning with randomized strategies; and Section 5 provides the lower bounds. 

Related work. In [17] . Pnueli and Rosner study the synthesis of reactive modules. In their 
framework, there is no game graph; instead, the environment and the objective are specified 
using an LTL formula. In [14], Kupferman and Vardi extend these results in two directions: 
they consider CTL* objectives and imperfect information. Again, no game graph, but a 
specification formula is given to the synthesis procedure. We believe that our setting, 
where a game graph is given explicitly, is more suited to fully and uniformly understand 
the role of imperfect information. For example, Kupferman and Vardi claim that imperfect 
information comes at no cost, because if the specification is given as a CTL (or CTL*) 
formula, then the synthesis problem is complete for Exptime (resp. 2Exptime), just as in 
the perfect-information case. These hardness results, however, depend on the fact that the 
specification is given compactly as a formula. In our setting, with an explicit game graph, 
reachability games of perfect information are PTiME-complete, whereas reachability games 
of imperfect information are ExPTiME-complete [19J. None of the above papers provide 
symbolic solutions, and none of them consider randomized strategies. 

It is known that for Partially Observable Markov Decision Processes (POMDPs) with 
boolean rewards and limit-average objectives the quantitative analysis (whether the value is 
greater than a specified threshold) is ExPTiME-complete [15] , However, almost winning is 
a qualitative question, and our hardness result for almost winning of imperfect-information 
games does not follow from the known results on POMDPs. We give in Section 5 a detailed 
proof of the hardness result of [19] for sure winning of imperfect-information games with 
reachability objectives, and we show that this proof can be extended to almost winning 
as well. To the best of our knowledge, this is the first hardness result that applies to the 
qualitative analysis of almost winning in imperfect-information games. 
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A class of semiperfect-mform&tion games, where one player has imperfect information 
and the other player has perfect information, is studied in [3]. That class is simpler than 
the games studied here; it can be solved in NP n coNP for parity objectives. 

2. Definitions 

A game structure (of imperfect information) is a tuple G = (L, Iq, E, A, O, 7}, where L is 
a finite set of states, Iq G L is the initial state, E is a finite alphabet, ACLxSxLis 
a set of labeled transitions, O is a finite set of observations, and 7 : O — > 2 L \% maps each 
observation to the set of states that it represents. We require the following two properties 
on G: (i) for all t G L and all a G E, there exists £' £ L such that (£, <r, £') G A; and (ii) the 
set {7(0) I o G O} partitions L. We say that G is a game structure of perfect information 
if = L and 7^) = {£} for all £ £ L. We often omit (0, 7) in the description of games of 
perfect information. For a G E and s Q L, let Post^(s) = {f £ L | I £ s : cr,f ) G A}. 

Plays. In a game structure, in each turn, Player 1 chooses a letter in E, and Player 2 resolves 
nondeterminism by choosing the successor state. A play in G is an infinite sequence tt = 
£qo-q£\ . . . o n -\£ n o n . . . such that (i) £q = Iq, and (ii) for all i > 0, we have (£i, ai,£i + \) G A. 
The prefix up to £ n of the play tt is denoted by vr(n); its length is |vr(n)| = n + 1; and its ias£ 
element is Last(7r(n)) = £ n . The observation sequence of 7r is the unique infinite sequence 
7~ 1 (7r) = 00O0O1 . . . a n -.\o n a n . . . such that for all i > 0, we have £i G 7(0^). Similarly, the 
observation sequence of 7r(n) is the prefix up to o n of 7~ 1 (7r). The set of infinite plays in 
G is denoted Plays(G), and the set of corresponding finite prefixes is denoted Prefs(G). A 
state £ G L is reachable in G if there exists a prefix p G Prefs(G) such that Last(p) = £. 
For a prefix p G Prefs(G), the cone Cone(p) = { n G Plays(G) | p is a prefix of tt } is the 
set of plays that extend p. The knowledge associated with a finite observation sequence 
r = 00CJ0O1CJ1 . . . <7 n _io n is the set K(r) of states in which a play can be after this sequence 
of observations, that is, K(r) = {Last(p) | p G Prefs(G) and 7 _1 (/o) = r}. 

Lemma 2.1. Let G = (L, Iq, E, A, 0, 7) 5e a game structure of imperfect information. For 
a G Sj £ G L, and p, // G Prefs(G) with p' = p ■ a ■ £, let oe G O be the unique observation 
such that £ G j(o e ). Then K(7~ 1 ( / o / )) = Post^(K(7" 1 (p))) n 7(0^). 

Strategies. A deterministic strategy in G for Player 1 is a function a : Prefs(G) — > X. 
For a finite set A, a probability distribution on A is a function k : A — > [0, 1] such that 
SaeA K ( a ) = 1- We- denote the set of probability distributions on A by V(A). Given 
a distribution k G 2?(^4), let Supp(/t) = {a £ A \ n(a) > 0} be the support of k. A 
randomized strategy in G for Player 1 is a function a : Prefs(G) — > f (S). A (deterministic 
or randomized) strategy a for Player 1 is observation-based if for all prefixes p, p' G Prefs(G), 
if 7 _1 (/o) = 7 _1 (p'), then = a(p'). In the sequel, we are interested in the existence 
of observation-based strategies for Player 1. A deterministic strategy in G for Player 2 
is a function (3 : Prefs(G) x E — > L such that for all p G Prefs(G) and all a G E, we 
have (Last(p), a, a)) G A. A randomized strategy in G for Player 2 is a function j3 : 
Prefs(G) xS^ V(L) such that for all p G Prefs(G), all a G E, and all £ G Supp(/3(p, a)), we 
have (Last(/)), a, £) G A. We denote by Ac, -A®, and Bg the set of all Player-1 strategies, 
the set of all observation-based Player-1 strategies, and the set of all Player-2 strategies in 
G, respectively. All results of this paper can be proved also if strategies depend on state 
sequences only, and not on the past moves of a play. 
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The outcome of two deterministic strategies a (for Player 1) and (3 (for Player 2) in 
G is the play it = £oo~o£i ■ ■ ■ o n -\£n^n . . . G Plays(G) such that for all i > 0, we have 
Oi = a(ir(i)) and £i + \ = (3(ir(i), Oi). This play is denoted outcome(G, a, (3). The outcome 
of two randomized strategies a (for Player 1) and (3 (for Player 2) in G is the set of plays 
7r = ^oco^i • • • o- n -i£ n a n . . . G Plays(G) such that for all i > 0, we have a(n(i))(ai) > 
and l3{Tt(i),ai)(li + i) > 0. This set is denoted outcome(G, a, (3). The outcome set of the 
deterministic (resp. randomized) strategy a for Player 1 in G is the set Outcome^G, a) of 
plays 7r such that there exists a deterministic (resp. randomized) strategy (3 for Player 2 
with 7r = outcome(G, a, /3) (resp. ir G outcome(G, a, (3)). The outcome sets for Player 2 are 
defined symmetrically. 

Objectives. An objective for G is a set </> of infinite sequences of observations and input 
letters, that is, <fi C (0 x A play 7r = £oo~o£i ■ ■ ■ G n ^\£ n o n . . . G Plays(G) satisfies the 

objective </>, denoted 7r |= (p, if 7 -1 (-7r) G (p. Objectives are generally Borel measurable: a 
Borel objective is a Borel set in the Cantor topology on {O x S) w [13J. We specifically 
consider reachability, safety, Biichi, coBiichi, and parity objectives, all of them Borel mea- 
surable. The parity objectives are a canonical form to express all w-regular objectives |21j . 
For a play ir = loo~o£\ . . . , we write lnf(-7r) for the set of observations that appear infinitely 
often in 7~ 1 (7r), that is, lnf(-7r) = {o € O \ £i G 7(0) for infinitely many i's}. 

• Reachability and safety objectives. Given a set T C O of target observations, the reach- 
ability objective Reach (T) requires that an observation in T be visited at least once, 
that is, Reach (T) = { £000^1 . . . G Plays(G) \ 3k > ■ 3o e T : £ k e 7(0) }. Dually, 
the safety objective Safe(T) requires that only observations in T be visited. Formally, 
Safe(T) = { £oaoho-i . . . G Plays(G) | VA; > • 3o G T : 4 G 7(0) }. 

• Biichi and coBiichi objectives. The Biichi objective Buchi(T) requires that an observation 
in T be visited infinitely often, that is, Buchi(T) = { ir | lnf(7r) H T / }. Dually, the 
coBiichi objective coBuchi(T) requires that only observations in T be visited infinitely 
often. Formally, coBuchi(T) = { vr | Inf(vr) C T }. 

• Parity objectives. For d G N, let p : O — > {0, 1, . . . , d} be a priority function, which maps 
each observation to a nonnegative integer priority. The parity objective Parity(p) requires 
that the minimum priority that appears infinitely often be even. Formally, Parity(p) = 
{ 7r I min{ p(o) I o G I nf (tt) } is even }. 

Observe that by definition, for all objectives cp, if tt \= <f> and 7~ 1 (-7r) = 7 _1 (7r'), then ir' \= cp. 

Sure winning and almost winning. A strategy Aj for Player i in G is sure winning for an 
objective eft if for all tt G Outcomej(G, Aj), we have it \= (p. Given a game structure G and 
a state £ of G, we write Ge for the game structure that results from G by changing the 
initial state to £, that is, if G = (L, Iq, S, A, 0, 7), then G^ = (L, £, E, A, 0, 7). An event is 
a measurable set of plays, and given strategies a and (3 for the two players, the probabilities 
of events are uniquely defined [22]. For a Borel objective <p>, we denote by Pr"'^(0) the 
probability that <p is satisfied in the game Gi given the strategies a and (3. A strategy a 
for Player 1 in G is almost winning for the objective (p if for all randomized strategies (3 
for Player 2, we have Pr^(0) = 1. The set of sure-winning (resp. almost-winning) states 
of a game structure G for the objective (p is the set of states £ such that Player 1 has a 
deterministic sure-winning (resp. randomized almost-winning) observation-based strategy 
in Gt for the objective (p. 
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01 02 03 04 

Figure 1: Game structure G. 

Theorem 2.2 (Determinacy). [TO] For all perfect-information game structures G and all 
Borel objectives (ft, either there exists a deterministic sure-winning strategy for Player 1 for 
the objective (f), or there exists a deterministic sure-winning strategy for Player 2 for the 
complementary objective Plays(G) \ <f>. 

Notice that deterministic strategies suffice for sure winning a game: given a randomized 
strategy a for Player 1, let a D be the deterministic strategy such that for all p G Prefs(G), 
the strategy a D (p) chooses an input letter from Supp(a(p)). Then Outcome! (G, a D ) C 
Outcomei(G, a), and thus, if a is sure winning, then so is oP . The result also holds for 
observation-based strategies. However, for almost winning, randomized strategies are more 
powerful than deterministic strategies as shown by Example 12.31 

Example 2.3. Consider the game structure shown in Figure[TJ The observations 01,02,03, 04 
are such that 7(01) = {£1}, 7(02) = {£2,^2}, 7(03) = {^3,^3}, and 7(04) = {£4}. The tran- 
sitions are shown as labeled edges in the figure, and the initial state is £\. The objective 
of Player 1 is Reach({o4}), to reach state £4. We argue that the game is not sure winning 
for Player 1. Let a be any deterministic strategy for Player 1. Consider the deterministic 
strategy f3 for Player 2 as follows: for all p G Prefs(G) such that Last(p) G 7(02), if a(p) = a, 
then in the previous round f3 chooses the state £2, and if a(p) = b, then in the previous 
round j3 chooses the state £' 2 . Given a and f3, the play outcome(G, a, (3) never reaches £4. 
Similarly, Player 2 has no sure winning strategy for the dual objective Safe({oi, 02, 03}). 
Hence the game is not determined. However, the game G is almost winning for Player 1. 
Consider the randomized strategy that plays o and b uniformly at random at all states. 
Every time the game visits observation 02, for any strategy for Player 2, the game visits 
£3 and £3 with probability ^, and hence also reaches £4 with probability ^. It follows that 
against all Player 2 strategies the play eventually reaches £4 with probability 1. 

Spoiling strategies. To spoil a strategy of Player 1 (for sure-winning), Player 2 does not 
need the full memory of the history of the play, he only needs counting strategies. We 
say that a deterministic strategy (5 : Prefs(G) x £ — > L for Player 2 is counting if for all 
prefixes p, p' G Prefs(G) such that \p\ = \p'\ and Last(p) = Last(p'), and for all a £ E, we 
have (3(p,a) = (3(p',a). Let B C G be the set of counting strategies for Player 2. The memory 
needed by a counting strategy is only the number of turns that have been played. This type 
of strategy is sufficient to spoil the non- winning strategies of Player 1. 

Proposition 2.4. LetG be a game structure of imperfect information and <f> be an objective. 
There exists an observation-based strategy a° G Aq such that for all (3 £ Be we have 
outcome(G, a", [3) G <f> if and only if there exists an observation-based strategy a° G A G 
such that for all counting strategies (3 C G Bq we have outcome(G, a°, (3 C ) G (j). 
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Proof. We prove the equivalent statement that: Va° G A° G -3(3 G Bq ■ outcome(G, a°, f3) G" <b 
iff Va° G jl£r • 3/3 c G B C G : outcome(G, a°, /3 C ) 0. The right implication (<— ) is trivial. 
For the left implication (—»■), let a° G .A^. be an arbitrary observation-based strategy for 
Player 1 in G. Let (3 G £>g be a strategy for Player 2 such that outcome(G, a°, (3) 4>. Let 
outcome(G, a", f3) = £qo-q£\ ■ ■ ■ <y n -\£ n a n . . . and define a counting strategy f3 c for Player 2 
such that V/9 G Prefs(G) • V<7 G S : if Last(/j) = and <r = <r n _i for n = |p|, then 

P c (p,a) = £ n , and otherwise /3 c (p,a) is fixed arbitrarily in the set Post^(Last(/9)). Clearly, 
(3 C is a counting strategy and we have outcome (G, a°, (3) = outcome(G, a°, f3 c ) and thus 
outcome(G,a°,/3 c ) <j). □ 

Remarks. First, the hypothesis that the observations form a partition of the state space 
can be weakened to a covering of the state space, where observations can overlap [10j . In 
that case, Player 2 chooses both the next state of the game I and the next observation o 
such that I G 7(0). The definitions related to plays, strategies, and objectives are adapted 
accordingly. Such a game structure G with overlapping observations can be encoded by 
an equivalent game structure G' of imperfect information, whose state space is the set 
of pairs (i,o) such that t G 7(0). The set of labeled transitions A' of G' is defined by 
A' = { ((£,o),a,(£',o')) I (£,a,£') G A } and 7 /_1 (i,o) = o. The games G and G' are 
equivalent in the sense that for every Borel objective 4>, there exists a sure (resp. almost) 
winning strategy for Player i in G for (p if and only if there exists such a winning strategy 
for Player i in G' for <\>. 

Second, it is essential that the objective is expressed in terms of the observations. 
Indeed, the games of imperfect information with a nonobservable winning condition are 
more complicated to solve. For instance, the universality problem for Biichi automata can 
be reduced to such games, but the construction that we propose in Section [3] cannot be 
used. More involved constructions a la Safra are needed [20J. 

3. Sure Winning 

First, we show that a game structure G of imperfect information can be encoded by 
a game structure G K of perfect information such that for every objective 4>, there exists a 
deterministic observation-based sure-winning strategy for Player 1 in G for <b if arid only if 
there exists a deterministic sure-winning strategy for Player 1 in G K for (f>. We obtain G K 
using a subset construction similar to Reif's construction for safety objectives [19]. Each 
state in G K is a set of states of G which represents the knowledge of Player 1. In the 
worst case, the size of G K is exponentially larger than the size of G. Second, we present a 
fixed-point algorithm based on antichains of set of states [10], whose correctness relies on 
the subset construction, but avoids the explicit construction of G K . 

3.1. Subset construction for sure winning. Subset construction. Given a game struc- 
ture of imperfect information G = (L, Iq, S, A, 0, 7), we define the knowledge-based subset 
construction of G as the following game structure of perfect information: 

G K = (£,{/ },S,A K ), 

where C = 2^\{0}, and (si,cx, S2) G A K iff there exists an observation o G O such that 
S2 = Post^(si) (~1 7(0) and S2 7^ 0. Notice that for all s G C and all a G E, there exists a set 
s' G C such that (s, cr, s') G A K . 
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A (deterministic or randomized) strategy in G is called a knowledge-based strategy. 
To distinguish between a general strategy in G, an observation-based strategy in G, and a 
knowledge-based strategy in G K , we often use the notations a,a°, and a K , respectively. 

Lemma 3.1. For all sets s G L that are reachable in G K , and all observations o G , either 
s C 7(0) or s fl 7(0) = 0. 

Proof. First, the property holds for s = {lo}, the initial state in G K as it is a singleton. 
Second, we show that the property holds for any successor s' of any state s in G K . Assume 
that (s,o~,s f ) G A K . Then we know that s' = Post^(s) n 7(0) for some o G O. Hence, 
s' C 7(0) and s' fl 7(0') = for all o' 7^ o since the set {7(0) | o G 0} partitions L. □ 

Abusing the notation, for a play ir = soCo s i • • • Cn-iSn&n • • • G Plays(G K ) we define its 
observation sequence as the infinite sequence 7 _1 (7r) = ooo-qOi . . . a n ^\o n a n ... of observa- 
tions such that for all i > 0, we have Sj C 7(0,). This sequence is unique by Lemma 13.11 
The play ir satisfies an objective i/)C(Ox S) w if 7 _1 (7r) G 0. 

The correctness of the subset construction G K is established by the following two lem- 
mas which generalize the result of [19] for safety objective to any kind of objective. For 
Lemma 13.31 the proof of [19J is not sufficient, since violation of a safety objective can be 
witnessed by a finite prefix of play, while general objectives need an infinite witness. 

Lemma 3.2. If Player 1 has a deterministic sure-winning strategy in G K for an objective 
<f>, then he has a deterministic observation-based sure-winning strategy in G for <f>. 

Proof. Let a K be a deterministic sure-winning strategy for Player 1 in G K with the objective 
<fi. Define a° a strategy for Player 1 in G as follows: for every p G Prefs(G), let ct°{p) = 
a K (p K ) where p K is defined from p = Iqo~qI\ . . .o n -\l n by p K = sqGqSi . . . a n -\s n where 
Si = K(7 _1 (£oCo^i • • • o~i~ili)) for each < i < n. Clearly, a° is a deterministic observation- 
based strategy as 7 _1 (p) = 7~ 1 (p / ) implies p K = p' K . 

By contradiction, assume that a° is not a sure-winning strategy for Player 1 in G 
with the objective <f>. Then there exists a play tt G Outcomei(G, a°) such that tt \/= <fi. 
Let 7T = Iqo-q£io-i . . . and consider the infinite sequence ir K = so<toSiCi . . . where Si = 
K(7 _1 (7r(f))) for each i > 0. We show that ir K G Outcomei(G K , a K ). First, we have 
s = K( 7 - 1 (^(0))) = K( 7 - 1 (4)) = {^0}. Second, for any i > 0, we have 8i = K( 7 - 1 (vr(i))) 
and by Lemma E] we have = Post^ (sj) fl 7(0) where o is such that £i + % G 7(0) and 
so (si, o~i, Sj+i) G A K . Third, by definition of a°, we have cij = a°(ir(i)) = a K (-7r K (i)). This 
entails that tt k G Outcomei(G K , a K ). 

Now, observe that trivially li G K(7 -1 (-7r(i))) for any % > 0, that is li G Sj and so 
Sin^f(oi) where Oj is the unique observation such that t- L G 7(0^). Hence, by Lemma l3~T| 
we have S{ C 7(0^). Consequently, 7 _1 (-7r K ) = 7~ 1 (7r) and thus 7r K ^= (j) which contradicts the 
fact that q k is a sure-winning strategy for Player 1 in G K with the objective <f>. Therefore, 
a° is a sure-winning strategy for Player 1 in G with the objective <f>. □ 

Lemma 3.3. If Player 1 has a deterministic observation-based sure-winning strategy in G 
for an objective <fi, then Player 1 has a deterministic sure-winning strategy in G K for </>. 

Proof. First, it is easy to show by induction that for every finite prefix of play p K = 
sqo~oSi . . . c n _is n in Prefs(G K ), there exists a prefix of play p = ^o^o^i • • • Cn-l^n i n Prefs(G) 
that generates p K , that is such that s» = K(7 _1 (£o<7o^i • • • Cj-i^j)) for each < i < n; and 
for all such prefix of play p' that generates p K , we have 7 _1 (p) = 7~ 1 (p') (by Lemma I2TT|) . 
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Now, let a° be a deterministic observation-based sure-winning strategy for Player 1 in 
G that is sure-winning for <fi. We construct a deterministic strategy a K for Player 1 in G K as 
follows: for every p K G Prefs(G K ), let a K (p K ) = a°(p) where p generates p K . By the above 
remark, a K is well-defined (it is independent of the choice of p since a° is observation-based). 

By contradiction, assume that a K is not sure-winning for Player 1 in G K with objective 
4>. Then, there exists a play 7r K G Outcomei(G K , a K ) with ir K ^= <p. 

We construct the DAG D = (V,E) where V = \ £ G Last(7r K (i))} and E = 

{((£,i),(£',i+l)) | (£,ai,£') G A}. By definition of G K , for all? > 0, we have Last(vr K (i)) ^0 
and for all I G Last(7r K (i)), there is a path in D from (^o, 0) to (£, i). Therefore, V is infinite 
and by Konig's Lemma, there exists an infinite path (£q,0)(£i,1) . . . in D and thus a play 
7r = ^oco^ici • • • hi G such that 7r G Outcomei(G, a°) and 7r ^ 4>. This is in contradiction 
with the assumption that a° is sure-winning in G for <fi. Hence a K is sure-winning for 
Player 1 in G K with objective <f>. □ 

Lemma 13.21 and Lemma 13.31 yield Theorem 13.41 

Theorem 3.4 (Sure-winning reduction). Player 1 has a deterministic observation-based 
sure-winning strategy in a game structure G of imperfect information for an objective 4> if 
and only if Player 1 has a deterministic sure-winning strategy in the game structure G K of 
perfect information for (j). 

3.2. Two interpretations of the /i-calculus. From the results of Section 13.11 we can 
solve a game G of imperfect information with objective <fi by constructing the knowledge- 
based subset construction G K and solving the resulting game of perfect information for the 
objective (f) using standard methods. For the important class of w-regular objectives, there 
exists a fixed-point theory — the /x-calculus — for this purpose [8J. When run on G K , these 
fixed-point algorithms compute sets of sets of states of the game G. An important property 
of those sets is that they are downward closed with respect to set inclusion: if Player 1 has 
a deterministic strategy to win the game G when her knowledge is a set s, then she also 
has a deterministic strategy to win the game when her knowledge is s' with s' C s. And 
thus, if s is a sure-winning state of G K , then so is s'. Based on this property, we devise a 
new algorithm for solving games of perfect information. 

An antichain of nonempty sets of states is a set q C 2 L \ {0} such that for all s, s' G q, 
we have s (£. s' . Let A be the set of antichains of nonempty subsets of L, and consider 
the following partial order on A: for all q, q' G A, let q C q' iff Vs G q ■ 3s' G q' : s C s'. 
For q C 2 L , define the set of maximal elements of q by [g] = {s G q \ s ^ and Ms' G 
q : s <jL s'}. Clearly, [g] is an antichain. The least upper bound of g, q' G A is q U q' = 
\{s | s G q or s G g'}] , and their greatest lower bound is gl~l q' = \{s fl s' | s G g and s' G g'} 
] . The definition of these two operators extends naturally to sets of antichains, and the 
greatest element of A is T = {L} and the least element is _L = 0. The partially ordered set 
(A, E, U, n, T, _L) forms a complete lattice. We view antichains of state sets as a symbolic 
representation of C-downward-closed sets of state sets. 

A game lattice is a complete lattice V together with a predecessor operator CP re : 
V — > V. Given a game structure G = (L, Iq, S, A, 0, 7) of imperfect information, and its 
knowledge-based subset construction G K = (£,{lo},T>,A K ), we consider two game lattices: 
the lattice of subsets («S, C, U, D,C, 0), where S = 2 C and CPre : S — > 5 is defined by 
CPre(g) = {s G C \ 3a G £ • Vs' G £ : if (s,a,s') G A K , then s' G g}; and the lattice of 
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Lattice of subsets 

[ ]f = {s e C | s c 7 ( )} 
[xjf=E(x) 

[^{)!}w]f=Hf{n}N! 

[pre(^)]f= CPre(Mf ) 

l{^}^if= {u}u\i=Ml [x ^ q] } 

Lattice of antichains 

\o\i= Mo)} 

[x\f= £(x) 

[pi{j|}w]H^{S}[^ 

[pre(^)]#= [CPreKM^ 

[^}*{;}i?k=iC]} 



antichains (A, E, U, n, {L}, 0), with the operator [CPre] : .4 — > *4 defined by |~CPre](g) = 
\{s G £ | 3a G £ • Vo G • 3s' € q : Post CT (s) n 7(0) C s'}] . 
The /U- calculus formulas are generated by the grammar 

tp ::= o|x|(/?V</?|(^A</?| pre(y?) | /«x.</? | z^x.c/? 

for atomic propositions o G O and variables rr. We can define —>o as a shortcut for 
Vo'eO\{o} °'- ^ variable is free in a formula tp if it is not in the scope of a quantifier 
\ix or i/x. A formula tp is closed if it contains no free variable. Given a game lattice V, a 
valuation £ for the variables is a function that maps every variable x to an element in V. 
For q G V, we write £ [a; 1— > q] for the valuation that agrees with £ on all variables, except 
that x is mapped to q. Given a game lattice V and a valuation £, each /x-calculus formula tp 
specifies an element [<£>]] £ of V, which is defined inductively by the equations shown in the 
two tables below. If ip is a closed formula, then [y]^ for any valuation £. The follow- 
ing theorem recalls that perfect-information games can be solved by evaluating fixed-point 
formulas in the lattice of subsets. 

Theorem 3.5 (Symbolic solution of perfect-information games). [8 J For every tv-regular 
objective eft, there exists a closed fj,-calculus formula /iForm(^), called the characteristic 
formula of <j), such that for all game structures G of perfect information, the set of sure- 
winning states of G for <fi is [/iForm(0)] <5 . 

Downward closure. Given a set q G 5, the downward closure of q is the set q[ = {s G C \ 
3s' G q : s C s'}. Observe that in particular, for all q G 5, we have q[ and \q~\i = q[. 
The sets ql, for q G S, are the downward-closed sets. A valuation £ for the variables in the 
lattice S of subsets is downward closed if every variable x is mapped to a downward-closed 
set, that is, £{x) = £(x)[. 

Lemma 3.6. All downward-closed sets q,q' G S satisfy \q f] q'~\ = \q] n \q'~\ and \q U g'] = 



ALG. FOR oj- REGULAR GAMES 



11 



Lemma 3.7. For all ^-calculus formulas cp and all downward-closed valuations £ in the 
lattice of subsets, the set \cpjf is downward closed. 

Proof. We prove this lemma by induction on the structure of cp. 

• if cp = o for o G O. It is immediate to show that [</?]f = [vlfl- 

• if cp = x for a variable x. We have [v?]f = £{x) which is downward closed by hypothesis. 

• if ip = <pi^ ^ | if2 and both [yijf and [v^lf are downward closed. Then we have 

Ml = I^ilf { n } I^ 2 lf anc ^ *he resm t follows from the fact that union and intersection 

of downward closed sets are downward closed. 

• if cp = pre(ipi) and [</?i]f is downward closed. We show that fcplf is downward closed. Let 
si £[</?lf and let S2 G C such that s% Q s\. Let us show that S2 G[y]f • By definition of 
CPre, since s\ G[j>re(y?i)]£, there exists a G £ such that for any s^, if (si, cr, s^) G A K then 
s' x Gj^ijf. Consider any s' 2 such that (s2, <r, s' 2 ) G A K . According to the definition of G K , 
we have s' 2 = Post CT (s2) H 7(0) 7^ for some o G C Now, let s[ = Post CT (si) n 7(0). Since 
S2 Q si, we have s 2 C and thus s[ / 0. Therefore (si,<r, s^) G A K and so G[</?i]f- 
As the latter set is downward closed, we also have s 2 G[v?i]f and thus S2 G|pre((/9i)]|. 

• if cp = vx.tpi and [yijf/ is downward closed for any downward closed valuation £'. By 
Tarski's theorem, {cpjf is one of the set in the infinite sequence qo, qi, ■ ■ ■ defined by qo = C 
and for every i > 1, q± =\<p\g i j. Since qo is downward closed, every qi (i > 1) is also 
downward closed by the induction hypothesis. 

• if cp = fix.cpi and [</?i]f/ is downward closed for any downward closed valuation £'. The 
proof is similar to the previous case. □ 

Lemma 3.8. For all fi-calculus formulas cp, and all downward- closed valuations £ in the 
lattice of subsets, we have |~ [^]f ] ^^l^-p where \£~\ is a valuation in the lattice of 
antichains defined by \£~\{x) = \£ (x)~\ for all variables x. 

Proof. We prove this by induction on the structure of cp. 

• if cp = o for o G O. The claim is immediate. 

• if cp = x for a variable x. We have |~[</?]f] = = Mf£]- 

• if ip = cp 1 ^ ^ and both [[<£i]f] = [^il"^-| and [[^If] =[9 5 2]f|-|- Using Lemma EH] 
and Lemma \'S.7\ we have successively: 

\Mf] = \yife{ n = 
Wfl{n}rMfl =I^#i{n}^#T 

• if cp = pre(cpx) and \{<pi]f\ =[<pi}fa. 

(1) We prove the inclusion [^^C \ [cpjf]. First, let s G [CPre] ([yilf^). We know 
that there exists a G £ such that Vo G O ■ 3s" G[(/?i]^ : Post CT (s) n 7(0) C s ". Since 
[[Vilf] = [v 9 i][£:] (induction hypothesis), it is clear that for such a, if (s,a, s') G A K , 
then there exists s" G |~ [<£>i]f] such that s' C s" . And since {cpijf is downward closed 
(by Lemma [377]) we have s' G[(/?i]f , so that s G CPre([</?i]f ) (and thus s G [</?]] f ). 
Second, we show that s is maximal in [<£>]f • By contradiction, assume that there 
exists si G[(/7]^ with s C sj. Then, by the same argument as in the first part of the 
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proof of the inclusion |~ [</?]f ] ^[^jA-, below, we have that s± satisfies the definition 
of [CPre] (|<^i]'r£-|) up to the operator [•]. This means that s is not maximal in 
[CPre] ([vil^-i)) a contradiction. 
(2) We prove the inclusion [~ [v?]f ] Qlflf £ y This is trivial if \l<p}f\ = 0. Otherwise, let 
us first show that [i^i]"^^ 0. Let s £ [ Mf]- Then, there exists a E S such that 
for any s', if (s, a, s') € A K then s' G[<^i]f • Since the transition relation of G is total 
and the observations partition the state space, we have Post CT (s) n 7(0) 7^ for some 

G O. Therefore, \<£i\f is nonempty and so is [yijrf-i • 

Now, we proceed with the proof of inclusion. Let s £ [ [</?]f ] , and let a 6 £ such that 
such that for any s', if (s,a,s') € A K then s' ^{^Pi}f- Let us show that s € [CPre 

1 (I^ilffi )• First, consider an arbitrary observation o G O and let s' = Post CT (s)ri7(o). 
We must show that there exists s" sjyijj^i such that s' C s". This is obvious if s' = 
since is nonempty. Otherwise, by the definition of G K , we have (s, a, s') € A K 
and therefore s' €[<^i]f • Since = |~[v?i]f] (induction hypothesis), there exists 
s" €f [yij^-i such that s' C s" , and thus s' satisfies the definition of [CPre] ({fij j^-i ) 
up to the operator [•] . 

Second, let us show that s is maximal in [CPre] ([^l] r^i )• By contradiction, assume 
that there exists si € [CPre] ([^ll'Tgi) with s C s±. Then, by the same argument as 
in the first part of the proof of the inclusion , we have s\ E[^]f • This 

implies that s |~ [v?]f ] , a contradiction. 

• if (p = vx.tpi and |" [<£>i]f/] =[^i]"^/-| for any downward closed valuation £'. By Tarski's 
theorem, [ipjf is one of the set in the infinite sequence qo,qi,... defined by go = £ and 
for every £ > 1, = [M]f r xt _»g- i]> ana - similarly, [yl'Ai is one of the set in the infinite 
sequence q' ,q'i,--- defined by q' Q = {L} and for all i > 1, % = l ( pj'f£y xl ^ q i y Observe 
that q' = [go]- By induction, assume that q' i _ 1 = \qi-i \ for some i > 1. Then q\ = \q{\ 
as \S ] [x 1 ^ g-_ x ] = [f [a; i-» . 

• if 9? = [ix.tpi and |~ [</?i]f/] = [¥?i]|^y-| for any downward closed valuation £'. The proof is 
similar to the previous case. □ 

Consider a game structure G of imperfect information and a parity objective (j). From 
Theorems 13.41 and 13.51 and Lemma 13.81 we can decide the existence of a deterministic 
observation-based sure-winning strategy for Player 1 in G for <fi without explicitly con- 
structing the knowledge-based subset construction G K , by instead evaluating a fixed-point 
formula in the lattice of antichains. 

Theorem 3.9 (Symbolic solution of imperfect-information games). Let G be a game struc- 
ture of imperfect information with initial state Iq. For every ui-regular objective <j>, Player 1 
has a deterministic observation- based strategy in G for <fi if and only if {Iq} Cj/iForm^)]^. 

Corollary 3.10. Let G be a game structure of imperfect information, let p be a priority 
function, and let £ be a state of G. Whether £ is a sure-winning state in G for the parity 
objective Parity(p) can be decided in Exptime. 
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Corollary 13.101 is proved as follows: for a parity objective eft, an equivalent /i-calculus for- 
mula tp can be obtained, where the size and the fixed-point quantifier alternations of cp is 
polynomial in <fi. Thus given G and 4>, we can evaluate <p in G K in Exptime. 

4. Almost Winning 

Given a game structure G of imperfect information, we first construct a game struc- 
ture H in which the knowledge of Player 1 is made explicit. However, the construction 
is different from the one used for sure winning. Then, we establish certain equivalences 
between randomized strategies in G and H. Finally, we show how the reduction can be 
used to obtain a symbolic Exptime algorithm for computing almost-winning states in G 
for Biichi objectives. An Exptime algorithm for almost winning for coBiichi objectives 
under imperfect information remains unknown. 

4.1. Subset construction for almost winning. Given a game structure of imperfect 
information G = (L, Iq, E, A, 0, 7), we construct the game structure H = Knw(G) = 
(Q,qo,T,, Ah) as follows: Q = { (s,£) \ 3o G O : s C 7(0) and £ G s }; the initial state is 
Qo = ({ ^0 }j ^0); the transition relation Ah Q Q x £ x Q is defined by ((s, £), a, (s 1 , £')) G Ah 
iff there is an observation o G O such that s' = Post^(s) n-y(o) and (£, a, if) G A. Intuitively, 
when H is in state (s,£), it corresponds to G being in state £ and the knowledge of Player 
1 being s. Two states q = (s, £) and q' = (V, £') of il are equivalent, written q ss </, if 
s = s', that is when the knowledge of Player 1 is the same in the two states. Two prefixes 
and p' = q' cr' q'i ■ ■ ■ a' n _ x ^ n of H are equivalent, written p p, if for 
all < % < n, we have qi ~ q[, and for all < i < n — 1, we have cr, = a\. Two plays 
7r, 7r' € Plays(lf) are equivalent, written 7r ~ 7r', if for all i > 0, we have 7r(i) ~ 7r'(z). For 
a state q £ Q, we denote by [q]~ = {q'£Q\q~q'} the ^-equivalence class of q. We 
define equivalence classes for prefixes and plays similarly. We cannot reuse the results of 
Section [3] to compute almost- winning states of G, as the randomized strategies in H should 
not distinguish equivalent states. 

Equivalence-preserving strategies and objectives. A strategy a for Player 1 in H is positional 
if it is independent of the prefix of plays and depends only on the last state, that is, for 
all p, p' G Prefs(-ff) with Last(/)) = Last(p'), we have a(p) = a(p'). A positional strategy a 
can be viewed as a function a : Q — > T>(E). A strategy a for Player 1 in H is equivalence- 
preserving if for all p, p' € Prefs(if) with p p' , we have a(p) = a(p'). We denote by Ah, 
A H , and A H the set of all Player-1 strategies, the set of all positional Player-1 strategies, 
and the set of all equivalence-preserving Player-1 strategies in H, respectively. We write 
A H ^ = A H H A H for the set of equivalence-preserving positional strategies. 

An objective <f> for H is a subset of (Q x S) w , that is, the objective is a set of plays. 
The objective 4> is equivalence-preserving if for all plays ir G (f>, we have [7r]~ C 0. 

Relating prefixes and plays. We define a mapping /i : Prefs(G) — > Prefs(ff) that maps 
prefixes in G to prefixes in as follows: given /? = £§oq£\o-\ . . . a n -\£ n , let h{p) = 
Qoo~oQio~i ■ ■ ■ &n-iqn, where for all < i < n, we have % = (si,ti), and for all < i < n — 1, 
we have Sj = K(7 _1 (/o(i))). The following properties hold: (1) for all p, pi G Prefs(G), if 
7 _1 (/)) = 7 -1 (//), then sa h(p'); and (ii) for all p, p' G Prefs(^T), if p w p', then 
7 _1 (/i _1 (p)) = 7~ 1 (/i~ 1 (//)). The mapping h : Plays(G) — > Plays(i^) for plays is defined 
similarly, and has similar properties. 
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Relating strategies for Player 1. We define two strategy mappings ~g : Ah —> A G and 
h : Ag — ► Ah- Given a Player-1 strategy an in H, we construct a Player-1 strategy 
a G = 9( a H) in G as follows: for all p G Prefs(G), let a G (p) = an{h{p)). Similarly, given a 
Player-1 strategy «g in G, we construct a Player-1 strategy a# = h(a G ) in -ff as follows: 
for all p G Prefs(-ff), let an(p) = ct G (h (/?))• The following properties hold: (i) for all 
strategies an G Af/> if is equivalence-preserving, then g(an) is observation-based; and 
(ii) for all strategies a G € Ass if «G is observation-based, then h(a G ) is equivalence- 
preserving. 

Relating strategies for Player 2. Observe that for all q G Q, all <r G S, and all £' G L, 
we have |{ s' \ (q,a,q') G A# A q' = (s',£') }[ < 1. Given a Player-2 strategy Ph in -H", 
we construct a Player-2 strategy /3 G = ~9~{Ph) as follows: for all p G Prefs(G), cr G E, 
and f G L, let /3 G (p, o-)(£') = p H (h(p),cr)(s' ,£') if for s = K( 7 - 1 ( / o)) and £ = Last(p), we 
have ((s,£),a, (s',£')) G A// for some (and then unique) s', and p G (p,a)(£') = otherwise. 
Similarly, given a Player-2 strategy P G in G, we construct a Player-2 strategy /3# = h{p G ) 
in as follows: for all p G Prefs(H), all a G S, and all (s,^), let Ph(p,o-)((s,£)) = 
p G (h- l {p),a){£). 

Lemma 4.1. T/ie following assertions hold. 

(1) For all ph G Prefs(iJ), /or every equivalence preserving strategy an, for every strategy 
Ph we have 

Pr^(Cone(p H )) = Prf o (a ^' l(fe) (^ 1 (Cone(p if ))). 

(2) For aZZ £ Prefs(G), for every observation-based strategy ao, for every strategy (3q 
we have 

Prf o G ^(Cone( PG )) = Pr^)^«)(/ l (Cone(p G ))). 

Proof. The following properties follow from the construction of strategies in Section 14.11 

(1) For all ph G Prefs(-ff), for every equivalence preserving strategy an, for every strategy 
Ph and for all p' H such that \p' H \ = \pn\ + 1, we have Pr° o H '^ H (Cone(p H ) \ Cone(pH)) = 

P^ aMf}H \h- l (Co n e(p> H )) | h- l (Cone( PH ))). 

(2) For all p G G Prefs(G), for every observation-based strategy a G , for every strategy (3 G 
and for all p' G such that \p' G \ = \p G \ + 1, we have Prf o G ' /?G (Cone(p G ) | Cone(p G )) = 

Prg aG) ^MCone(p G )) | k(Cone(p G ))). 

The proof for the first part is as follows: observe that 

Pr^(Cone(g )) = Prg^'^^^Conetgo))) 

= Prf o (QH) ' 5(te) (Cone(/ )) = l. 

The result follows from the above property and induction. The second part follows analo- 
gously. □ 

Theorem 4.2 (Almost-winning reduction). Let G be a game structure of imperfect in- 
formation, and let H = Knw(G). For all Borel objectives <f> for G, all observation-based 
Player-1 strategies a G in G, and all Player-2 strategies fi G in G, we have Pr^ G '^ G (c/>) = 

Prqg aG ^' h ^ G \h((/))) . Dually, for all equivalence-preserving Borel objectives (j) for H, all 
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Figure 2: Memory and randomization are necessary to almost win the objective Buchi({o4}). 



equivalence-preserving Player-1 strategies an in H, and all Player-2 strategies /3jj ^ n H , 
we have Pr^f '^(0) = Pr^ aMM (h' 1 ^)). 

Proof. By the Caratheodary unique-extension theorem, a probability measure defined on 
cones has a unique extension to all Borel objectives. The theorem then follows from 
Lemma 14.11 □ 

Corollary 14.31 follows from Theorem I4.2L 

Corollary 4.3. For every Borel objective $g for G, we have 

sup inf Pr" G ' /?G ($ G ) = sup inf Pr"f > fe (/i($ G )); 

3a G £A°.Vl3 G eB G : Pr^ G (d> G ) = 1 

iff3a H e A%. V(3h G B h : Pr£"^(/i(* G )) = 1. 



4.2. Almost winning for Biichi objectives. We first illustrate the need of memory and 
randomization for almost-winning in imperfect information games with Biichi objectives. 

Example 4.4 (Memory is needed to almost- win). Consider the example of Figure [2j The 
objective of Player 1 is to reach a state with observation 04. 

We show that Player 1 has no observation-based sure-winning strategy in this game. 
This is because when we fix an observation-based strategy for Player 1, Player 2 has a 
spoiling strategy to maintain the game into the states {£o,£i,£2}- Indeed, at £q, the only 
reasonable choice for Player 1 is to play a. Then Player 2 can choose to go either in £\ or 
£2- In both cases, the observation will be the same for Player 1. After seeing o\ao2, if the 
strategy of Player 1 is to play a then Player 2 chooses £2, otherwise, if Player 1 strategy 
is to play b then Player 2 chooses £\. This can be repeated and so Player 2 has a spoiling 
strategy against any observation-based strategy of Player 1. 

We now show that almost-winning strategies exist for Player 1. Consider that Player 1 
plays an observation-based randomized strategy a as follows: after a sequence of observa- 
tions r, 

• if Last(r) = o\, then a(r)(a) = 1 and a(r)(6) = 0, 

• if Last(r) = 02, then a(r)(a) = 0.5 and a(r)(b) = 0.5, 

• if r = t' ■ 01 ■ a ■ 03, then a{r)(a) = and a{r)(b) = 1, 
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• if r = t' ■ 02 ■ <J ■ 03, then a(r)(a) = 1 and o:(t)(6) = 0, 

• otherwise take, arbitrarily, a(r)(a) = 1 and a(r)(b) = 0. 

The strategy a is almost-winning against any randomized strategy of Player 2. Note 
that the strategy a uses memory and this is necessary because when receiving observation 
03, Player 1 has to play a if the previous state satisfied observation 01 and b if the previous 
state satisfied 02. 

Given a game structure G of imperfect information, let H = Knw(G). Given a set 
T C Oof target observations, let B r = { (s, t) G Q \ 3o G T : s C 7(0) }. Then 
/i(Buchi(T)) = Buchi( J B r ) = {tt h G Plays(#) | Inf(7r jf /)n5 r 7^ 0}. We first show that almost 
winning in H for the Biichi objective BucW\(Bq-) with respect to equivalence-preserving 
strategies is equivalent to almost winning with respect to equivalence-preserving positional 
strategies. Formally, for Bq- C Q, let Q^ s = { q G Q \ 3an G A h ■ V/3h G B h • Vg' G [g]~ : 

Pr^' fe (Buchi (B r )) = 1 }, and Q^ p) = { q G Q | 3a H G ^ (P) • Vftr G £^ • V?' G : 

Pr^' fe (Buchi( J B r )) = 1 }. We will prove that = Qas^- Lemma S3] follows from the 
construction of H from G, and yields Lemma 14.61 

Lemma 4.5. For a// q\ G Q, arcd aZ/ it 6 S, if (<7i>c ; Qi) £ Ajy, f/jen /or a// G [<Zi]«> 
there exists q2 G [gi]?« such that (q2,o-,q' 2 ) ^ 

Lemma 4.6. Given an equivalence-preserving Player-1 strategy an G »4f/> a prefix p G 
Prefs(ff) ; and a state q G Q, if there exists a Player-2 strategy f3jj G Bh such that 
■p v <XH,PH (Cone(p)) > ; then for every prefix p' G Prefs(-ff) with p ~ p' , there exist a 

Player-2 strategy f3' H G Bh and a state q' G [g]~ such that Pr^, H '^ H (Cone(//)) > 0. 

Observe that Q\Q% 5 = {q G Q \ Vajy G A H -3(3 H G B^-3g' G : Pr^' fe (Buchi(£ r )) < 
1 }. It follows from Lemma [4. 61 that if a play starts in Q^ s and reaches Q \ Q^s with positive 
probability, then for all equivalence-preserving strategies for Player 1, there is a Player 2 
strategy that ensures that the Biichi objective Buchi(i^r) is satisfied with probability strictly 
lower than 1. 

Notation. For a state q G Q and Y C Q, let Allow(g, Y) = { a G £ | Post^ (g) C7}. For a 
state g G Q and F C Q, let Allow([g]~, Y) = f\'e[ 9 ] ra Allow(g', Y). 

Lemma 4.7. For a// g G Q^S' ^ ^ aue Allow([g]«, Q^s) 7^ ^- 

Proof. Assume towards contradiction that there exists g G Q^s such that Allow([g]~, Q^s) = 
0. Then for all a G £ there exists g' G [g]~ such that Post^ (g') n (Q \ Q^s) ^ ^ Hence for 
every equivalence preserving strategy there exists q' G [g]~ such that a# (g')(o") > and 
Post^ (g') n (Q \ QXs) + 0- Hence for every equivalence strategy an there is a state q' G [g]~ 
and a strategy for Player 2 such that Q \ QXs ^ s reached with positive probability. This 
contradicts that [g]~ C Q^s- □ 

Lemma 4.8. Given a state q G Q^S' ^ a # ^ be an equivalence-preserving Player-1 
strategy such that for all Player-2 strategies Ph G Bh and all states q' G [g]«, we have 
Pr" / iI ^ H (Buchi(i?7-)) = 1. Let p = gotfogi . . . a n -iq n be a prefix in Preh(H) such that for all 
< i < n, we have qi G Q%$- If there is a Player-2 strategy (3h G Bh and a state q' G [g]~ 
such that Pr^"' 13 " (Cone(p)) > 0, then Supp(a#(yo)) C Allow([g n ]~, Q% s ). 
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Rank(2) Rank(l) Rank(O) 



Figure 3: Game structure H = Knw(G) (for G of Figured]). 



Proof. Fix an almost-winning strategy an- Assume towards contradiction for a history pu 
satisfying the conditions of the lemma that there exists a G Supp(a#(p#))\Allow([(/ n ], Q^)- 
Then there exists q' n G [q n ]x such that Post^ (q' n ) n (Q \ Qas) ^ ®- Then there exists p' H 
such that ph ~ p'h and Last(p^-) = q' n . Then by Lemma B~6l there exists a strategy (5' H and 

g' G such that Vx^f 1 ' H (Cone(p' H )) > 0. Then given p' H and the strategy olh there exists 
a Player 2 strategy such that Q \ Q^ s is reached with positive probability. This contradicts 
that an is an almost-winning strategy. □ 

Notation. We inductively define the ranks of states in as follows: let Rank(0) = 
Br n Q% s , and for all j > 0, let Rank(j + 1) = Rank(j) U { q G Q% 5 | 3cr G Allow([g]~, Q£ s ) : 
Post^(g) C Rank(j)}. Let j* = min{j > | Rank(j) = Rank(j + 1)}, and let Q* = Rank(j*). 
We say that the set Rank(j + 1) \ Rank(j) contains the states of rank j + 1, for all j > 0. 

Example 4.9. Given the game structure G of imperfect information from Figure [U the 
game structure H = Knw(G) is shown in Figure EJ All states are almost winning for the 
Biichi objective Buchi({ ({£4}, £4) }). The ranks of the states are shown next to the states. 
The positional strategy that plays both a and b with equal probability is almost winning 
at all states. For the states q with ranks 1, 3, and 4, if the rank of q is j, then Post^(g) C 
Rank(j - 1) and Postf (q) C Rank(j - 1). For the states with rank 2, if q = ({ £ 2 ,£' 2 },£ 2 ), 
then Post^(g) C Rank(l); and if q = ({ £ 2 ,£ 2 },£ 2 ), then Postf (q) C Rank(l). 

Lemma 4.10. Q* = Q% s . 

Proof. By definition, Q* C Q^s- We now prove that Qas — Q* ■ Assume towards a 
contradiction that X = \ Q* 7^ 0- For all states q G X and all a G Allow([g]~, Q^s)' we 
have Post^ (g)nl/ 0, because otherwise g would have been in Q* . Hence, for all q G X 
and all a G Allow([g]~, Q%s), there exists a?'£l such that (q, a, q') G A#. Fix a strategy 
for Player 2 as follows: for a state } £ I and the input letter a G Allow([g]~, Q^s)' 
choose a successor q' £ X such that (q,a,q') G A//. Consider a state q £ X and an 
equivalence-preserving almost-winning strategy a// for Player 1 from q for the objective 
BucW\(Bq-). By Lemma 14,8} for every prefix p satisfying the condition of Lemma 14.8} we 
have Supp(Q H (p)) C Allow([Last(p)]«, Q% s ). It follows that Pr^' fe (Safe(X)) = 1. Since 
Br^QXs ^ Q*> it; follows that B r nX = 0. Hence Pr^> fe ( Reach (Br)) = 0, and therefore 
~PYq H 'P H (BucW\(Bq-)) = 0. This contradicts the fact that an is an almost-winning strategy. 

□ 
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Equivalence-preserving positional strategy. Consider the equivalence-preserving positional 
strategy oF H for Player 1 in H, which is defined as follows: for a state q G Qas> cnoose au 
moves in Allow([g]~, Q%$) uniformly at random. 

Lemma 4.11. For all states q G Q%$ and all Player-2 strategies (3h in H, we have 
Pr^' fe (Safe(Q~ s )) = 1 and Pr^' fe (Reach [B r n Q% s )) = 1. 
Proof. By Lemma f4. 101 we have Q* = Q^ s - Let z = \Q*\- 

• For a state q G QaS' we have Post^(g) C Q^ s for all a G Allow([g]~, Qas)- ^ follows for 

all states q G Q^s anc ^ au strategies for Player 2, we have Pr g H ' (Safe(Q A i s )) = 1. 

• For a state q G (Rank(j + 1) \ Rank(j')), there exists a G Allow([g]~, Q^s) such that 
Post^((?) C Rank(j). For a set Y C Q, let J (y) denote the set of prefixes that reach Y 
after at most j steps. It follows that for all states q G Rank(j + 1) and all strategies (3h 
for Player 2, we have 

Pr^COHRanktf))) > J-. 

Let -B = H QXs- By induction on the ranks it follows that for all states q G Q* and 
all strategies (5 for Player 2: 

Prf"' fe (0 2 (Rank(0))) = ¥rf^ H (0 Z (B)) > (±Y = r > 0. 

For m > 0, we have Pr^' fe (0 m ' 2 (5)) > 1 - (1 - r) m . Thus: 

Pr^' fe (Reach( J B)) = lim Prf 11 ' 13 " (0 m ' z (B)) > lim 1 - (1 - r) m = 1. 

m— >oo m— >oo 

The lemma follows. □ 

Lemma [4. 1 II implies that, given the Player- 1 strategy otP H , the set Q^s is never left, and 
the states in Br D Q^s are reached with probability 1. Since this happens for every state in 
Q%$, it follows that the set Bq- Pi Q^s is visited infinitely often with probability 1, that is, 
the Biichi objective Buchi(i?7-) is satisfied with probability 1. This analysis, together with 

the fact that [qo]~ is a singleton and Corollary 14.31 proves that QX S = QaP- Theorem E2] 
follows. 

Theorem 4.12 (Positional almost winning for Biichi objectives under imperfect informa- 
tion). Let G be a game structure of imperfect information, and let H = Knw(G). For 
all sets T of observations, there exists an observation-based almost-winning strategy for 
Player 1 in G for the objective Buchi(T) iff there exists an equivalence-preserving positional 
almost- winning strategy for Player 1 in H for the objective Buchi(£>3-) . 

Symbolic algorithm. We present a symbolic quadratic-time (in the size of H) algorithm 
to compute the set Q% s . For Y C Q and X C Y, let Apre(yX) = { q G Y \ 3a G 
Allow([g]»,y) : Post^(g) C X }; and Spre(y) = { q G Y | Allow([g]«,y) ^ }. Note that 
Spre(y) = Apre(y,y). Let 

cp = uY.fiX. (Apre(yX) V (B T A Spre(y)) 

and let Z =[</>]. 
Lemma 4.13. Z = Q^ s - 
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Proof [of Lemma l4.13j . We prove Z = by proving inclusion in both directions. We 
have Z =[</>] and <p = vY.pX. (Apre(F, X) V {B T A Spre(F))) 

(1) We first show that Z C Q^s- Since Z is a fixed-point of <j> we have 

Z ={nX.(Apre(Z,X) V (B T A Spre(Z)))] . 

We analyze the evaluation of Z as the fixed-point as follows: let Xq = and Xj+i = 
Apre(Z, Xi) V (#r A Spre(Z)). Observe that since X = we have Apre(Z, X ) = and 
hence X\ = Bq- n Spre(Z) C Let j* = min{ i | Xj+i = Xi} and we have Z = Xj*. 
Consider the equivalence preserving strategy for Player 1 that at a state q € Z plays 
all moves in Allow([g]~, Z) uniformly at random. For all q G Z, for all q' € and 
for all a € Allow([(7]~, Z), we have Post^(g') C Z. It follows that for all strategies (3h 

for Player 2 and for all states q £ Z we have Pr^^' (Safe(Z)) = 1. Also for a state 
q S (-2Q-t_i \ Xi) \ Br we have there exists a E A\\ow({q]~, Z) such that Post^ (q) C Xj, 
i.e., for a state g € (Xj+i \Xj) \.Bt> given against all strategies (3h the next state is 
in Xi with probability at least t^t. Arguments similar to Lemma 14. Ill establishes that 
c? H is an almost-winning strategy for all states q € Z . Hence we have Z C QX$- 

(2) We now show that Q^c, Q Z. We first show that Q^c, satisfies that 

QX S =lfxX.(Apre(Q% s ,X) V (Br A Spre(Q£ s )))] . 

Observe that Q^s = Spre(Q^s). We now analyze the evaluation of Qas as the fixed- 
point Q* as shown in Lemma f4. 101 Let Xq = 0, then Apre(<5^ s , Xq) = 0. Hence X\ = 
Apre(Q^ s ) V (B T A Spre(Q^ s )) = B r A Q% s = Rank(0) (as defined before LemmaHTD]). 
By the definition of Rank(j + 1) from Rank(j) and the definition of Apre(-, •) and Spre(-) 
it follows that for all i > 0, given Rank(z — 1) = Xi, we have Rank(z) = Xj + i = 
Apre(Q^ s , Xi) V {Br A Spre(Q^s))- By induction we have 

Q* =[//X.(Apre(Q£ s , X) V (B r A Spre(Q~ s )))] . 

Since Q* = Q^s (by Lemma [4. 10[) we obtain the desired result. Since Z is the greatest 
fixed-point we have Q^s C Z. 
The result follows. □ 

Theorem 4.14 (Complexity of almost winning for Biichi objectives under imperfect infor- 
mation). Let G be a game structure of imperfect information, letT be a set of observations, 
and let £ be a state of G. Whether t is an almost-winning state in G for the Biichi objective 
Buchi(T) can be decided in Exptime. 

The facts that Z = Q^s an< ^ that H is exponential in the size of G yield Theorem 14.141 
The arguments for the proofs of Theorem 14.121 and 14.141 do not directly extend to coBiichi 
or parity objectives. In fact, Theorem 14.121 does not hold for parity objectives in general, 
for the following reason: in concurrent games with parity objectives with more than two 
priorities, almost-winning strategies may require infinite memory; for an example, see [5]. 
Such concurrent games are reducible to semiperfect-information games [I] , and semiperfect- 
information games are reducible to the imperfect-information games we study. Hence a 
reduction to finite game structures of perfect information in order to obtain randomized 
positional strategies is not possible with respect to almost winning for general parity objec- 
tives. Theorem 14. 121 and Theorem 14.141 may hold for coBiichi objectives, but there does not 
seem to be a simple extension of our arguments for Biichi objectives to the coBiichi case. 
The results that correspond to Theorems 14. 121 and 14. 141 for coBiichi objectives are open. 
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Direct symbolic algorithm. As in Section 13.21 the subset structure H does not have to 
be constructed explicitly. Instead, we can evaluate a fixed-point formula on a well-chosen 
lattice. The fixed-point formula to compute the set Q^s is evaluated on the lattice (2®, C 
, U, Pi, Q, 0). It is easy to show that the sets computed by the fixed-point algorithm are 
downward closed for the following order on Q: for (s,£), (s',£') G Q, let (s,£) ^ (s',£') 
[&£ = £' and s C s'. Then, we can define an antichain over Q as a set of pairwise 
incomparable elements of Q, and compute the almost-sure winning states in the lattice of 
antichains over Q, without explicitly constructing the exponential game structure H. 

5. Lower Bounds 

We show that deciding the existence of a deterministic (resp. randomized) observation- 
based sure-winning (resp. almost-winning) strategy for Player 1 in games of imperfect infor- 
mation is ExPTiME-hard already for reachability objectives. A first proof for sure-winning 
was given in [19]. We give all the details of the reduction used in the proof and show that 
it extends to almost winning as well. 

Sure winning. To show the lower bound result, we use a reduction of the membership 
problem for polynomial space Alternating Turing Machine. An alternating Turing machine 
(ATM) is a tuple M = (Q, qo, g, Ej, S t , 5, F) where: 

• Q is a finite set of control states; 

• Qo G Q is the initial state; 

• g : Q -» {A,V}; 

• Sj = {0, 1} is the input alphabet; 

• = {0, 1, 2} is the tape alphabet and 2 is the blank symbol; 

• ^CQxSjxQxS(X {-1, 1} is a transition relation; and 

• F C Q is the set of accepting states. 

We say that M is a polynomial space ATM if for some polynomial p(-), the space used by 
M on any input word w is bounded by p( [«;[). 

Without loss of generality, we make the hypothesis that the initial control state of the 
machine is a V-state and that transitions link V-state to A-state and vice versa. A word 
w is accepted by an ATM M if there exists a run tree of M on w whose all leaf nodes are 
accepting configurations (see [3] for details). The AND-OR graph of the polynomial space 
ATM (M,p) on the input word w G E* is G(M,p) = (S v , S A , s ,=>, R) where 

• S y = {(q, h,t)\qeQ, g(q) = V, 1 < h < p(\w\) and t G £f (H) }; 

• 5 A = {(q, h,t)\q€Q, g(q) = A, 1 < h < p(\w\) and t G £? (H) }; 

• s o = (<70) 1) t) where t = lo.Ef , 

• ((lii hi, h), (q2, h2, t2)) G=^ iff there exists (gi, t\(h\), q, 7, d) G 5 such that q2 = q, /12 = 
h\ + d, t2(h\) = 7 and t2(i) = ti(i) for all i 7^ hi; 

• R={(q,h,t) eS v uS A \geF}. 

A word w is accepted by (M,p) iff R is reachable in G(M,p). The membership problem 
is to decide if a given word w is accepted by a given polynomial space ATM (M,p). This 
problem is known to be ExpTime-hard [3]. 

Idea of the reduction. Given a polynomial space ATM M and a word w, we construct a 
game of size polynomial in the size of (M, w) to simulate the execution of M on w. Player 1 
makes choices in V-states and Player 2 makes choices in A-states. Furthermore, Player 1 is 
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responsible for maintaining the symbol under the tape head. The objective is to reach an 
accepting configuration of the ATM. 

Each turn proceeds as follows. In an V-state, by choosing a letter (t, a) in the alphabet 
of the game, Player 1 reveals (i) the transition t of the ATM that he has chosen (this way 
he also reveals the symbol that is currently under the tape head) and (ii) the symbol a 
under the next position of the tape head. If Player 1 lies about the current or the next 
symbol under the tape head, he should loose the game, otherwise the game proceeds. The 
machine is now in an A-state and Player 1 has no choice: he announces a special symbol e 
and Player 2, by resolving nondeterminism on e, chooses a transition of the Turing machine 
which is compatible with the current symbol under the tape head revealed by Player 1 at 
the previous turn. The state of the ATM is updated and the game proceeds. The transition 
chosen by Player 2 is visible in the next state of the game and so Player 1 can update 
his knowledge about the configuration of the ATM. Player 1 wins whenever an accepting 
configuration of the ATM is reached, that is w is accepted. 

The difficulty is to ensure that Player 1 looses when he announces a wrong content of 
the cell under the tape head. As the number of configurations of the polynomial ATM is 
exponential, we cannot directly encode the full configuration of the ATM in the states of 
the game. To overcome this difficulty, we use the power of imperfect information as follows. 
Initially, Player 2 chooses a position k, 1 < k < p(\w\), on the tape: this number as well 
as the symbol a £ {0, 1, 2} that lies in the tape cell number k is maintained all along the 
game in the non-observable portion of the game states. The pair (a, k) is thus private to 
Player 2 and invisible to Player 1. Hence, at any point in the game, Player 2 can check 
whether Player 1 is lying when announcing the content of cell number k, and go to a sink 
state if Player 1 cheats (no other states can be reached from there). Since Player 1 does 
not know which cell is monitored by Player 2 (A; is private), to avoid loosing, he should not 
lie about any of the tape cells and thus he should faithfully simulate the machine. Then, 
he wins the game if and only if the ATM accepts the words w. 

Almost winning. To establish lower bound for almost-winning, we can use the same reduc- 
tion. Randomization can not help Player I in this game. Indeed, at any point of the game, 
if Player I takes a chance in either not faithfully simulating the ATM or lying about the 
symbol under the tape head, the sink state is reached. In those case, the probability to 
reach the sink state is positive and so the probability to win the game is strictly less than 
one. We now present the details of the reduction of the hardness proof. 

Reduction. Given a polynomial space ATM (M,p), with M = (Q, qo, g, Ej, E t , 5, F) and a 
word w, we construct the following game structure Gm,p,w = (L, lo, E, A, 0, 7}, where: 

• The set of positions L = {init} U {sink} U L\ U L 2 where: L\ = (5 U {-}) x Q x 
{1, . . . ,p(|tu|)} x {1, . . . ,p(|u;|)} x E^. A state (t, q, h, k, a) consists of a transition t € 5 
of the ATM chosen by Player 2 at the previous round or — if this is the first round where 
Player 1 plays, the current control state q of M, the position h of the tape head, the pair 
(k, a) such that the k-th symbol of the tape is a, this pair (k, a) will be kept invisible 
for Player 1. L 2 = Q x {1, . . . ,p(\w\)} x T, t x {1, . . . ,p(\w\)} x T, t . A state (q, h, 7, k, a) 
consists of q, h, k, a as in L\ and 7 is the symbol that Player 1 claims to be under the 
tape head. The objective for Player 1 will be to reach a state I G L associated with an 
accepting control state of M. 

• Iq = init. 

• E = {e} U (5 x E t ). 
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• The transition relation A contains the following sets of transitions: 

- I\ that contains transitions (init, e, (— , go, 1, k, a)) where (i) 1 < k < p(\w\) and (ii) a = 
w(k) if 1 < k < \w\ and a = 2 otherwise. I2 that contains transitions (init, (t, 7), sink) 
where (£,7) G 5 x Sj. So, at the initial state init, Player 1 has to play e in order to 
avoid entering sink. By resolving nondeterminism on e, Player 2 chooses a tape cell to 
monitor. 

- S that contains transitions (sink, a, sink) for all a G S. When the sink state is entered, 
the game stays there forever. 

- L\.\ that contains transitions (li,e, sink) for all i\ G L\\ L1.2 that contains transitions 
((t,q,h,k,a), {(q x , 71, g 2 , 72, d), 73), sink) where qi q or -.(1 < /i+ci < p(|w|)); L1.3 con- 
tains the transitions ((i, g, /i, A;, cr), ((gi , 71, g2, 72, d), 73), sink) where /i = fcA7i 7^ a or /i+ 
d = k A 73 7^ <r; L1.4 contains the transitions ((t, q, h, k, a), ((gi , 71, 52, 72, d), 73), (q2, h + 
d, 73, k, a')) such that q = qi, 1 < h + d < p(\w\), h = k — ► (71 = a A a' = 72), 
and h ^ k — > a' = a. Those transitions are associated with states of the game where 
Player 1 chooses a transition of the ATM to execute (if he proposes e, the game evolves 
to the sink state, see Li.i)- The transition proposed by Player 1 should be valid for 
the current control state of the ATM and the head should not exit the bounded tape 
after execution of the transition by the ATM, otherwise the game evolves to the sink 
state, see Li.2- When choosing a letter, Player 1 also reveals the current letter under 
the tape head (given by the transition) as well as the letter under the next position of 
the tape head. If one of those positions is the one that is monitored by Player 2, the 
game evolves to the sink state in case Player 1 lies, see Li.3,Li.4. 

- L2.1 contains the transitions ((q, h, 71, k, a), e, ((gi, 72, t/2, 73, d), q%, h+d, k, a')) such that 
q = qi, q2 = <?3, 7i = 72, 1 < h + d < p(\w\), h = k -> a' = 73, and h / k -> a' = a; 
L2.2 contains the transitions ((</, h, 71, k, <r),e,sink) such that there does not exist a 
transition (q, 71, qi, 72, d) G 5 with 1 < h + d < p(\w\); L2.3 contains the transitions 
((g, /i,7i,fc, a), (t, 7), sink) where (£,7) G S \ {e}. Those transitions are associated with 
states of the game where Player 2 chooses the next transition of the ATM to execute. 
Player 1 should play e otherwise the game goes to the sink state (see £2.3)- Also the 
game goes to the sink state if there is no valid transition to execute in the ATM (see 
L2.2). In the other cases, when Player 1 proposes e, Player 2 chooses a valid transition 
by resolving nondeterminism. The copy of the monitored cell is updated if necessary. 

• O = {init, sink} U Oi U 2 where Oi = {(t, q, h) \ 3(t, q, h, k, a) G L ± } and 0\ = {(q, h, 7) | 
3(g,/i,7, k,a) G L 2 }. 

• 7 is defined as follows: 7(init) = {init}, 7(sink) = {sink}, for all (t,q,h) G Oi, j(t, q, h) = 
{(t,q,h,k,a) G Li}, for all (q,h,y) G 2 , 7(9,^,7) = {(q,h,j,k,a) G L 2 }. 

Finally, the objective 4> of this game for Player 1 is to reach a state where the associated 
control state of the ATM is accepting, i.e. cp = {01O2 ■ ■ ■ o n ■ ■ ■ G O w \ 3i > : (oj = 
(t,q,h) e0 1 Vo i = (g,/i, 7 ) G 2 ) A q G F}. 

It follows that Player 1 has an observation-based sure-winning (or almost-winning) 
strategy in the game Gm,p,w for the objective <fi iff the word w is accepted by the polynomial 
space ATM (M,p). This gives us Lemma 15. II and Theorem 15.21 follows from the lemma. 

Lemma 5.1. Player 1 has a deterministic (resp. randomized) observation-based sure- 
winning (resp. almost-winning) strategy in the game Gm,p,w f or the objective (f> iff the 
word w is accepted by the polynomial space ATM (M,p). 
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Theorem 5.2 (Lower bounds). Let G be a game structure of imperfect information, let T 
be a set of observations, and let I be a state of G. Deciding whether I is a sure-winning 
state in G for the reachability objective Reach (T) is ExPTiME-Ziard Deciding whether £ is 
an almost-winning state in G for Reach (T) is also ExPTlME-hard. 
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